Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate test and prod cache (+ ruff formatter) #1789

Merged
merged 2 commits into from
Oct 31, 2023

Conversation

Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Oct 31, 2023

At the moment, tests running locally can alter huggingface_hub cache. Two majors cases happen:

  • either some "dummy" files/repos are downloaded to the cache
  • either the saved token is replaced by the CI one (or simply deleted).

I previously tried to fix this problem using fixtures to use a separate cache and save/set back the registered token while running tests. But this is not really satisfying since cleanup fixtures might not be processed (if process is killed or if several pytests processed are run at once). This PR aims to definitely get rid of these problems by setting a test HF_HOME when running the tests. By doing so, there is no way staging and prod caches get mixed or corrupted.

Looks like something I should have done long ago. Sorry for the inconveniences @McPatate 😄


Also I've been a bit too quick when replacing black with ruff in #1783. I fixed the makefile and github workflows in this PR to run both the linter and the formatter.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 31, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Member

@McPatate McPatate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌🏻

@Wauplin
Copy link
Contributor Author

Wauplin commented Oct 31, 2023

Thanks for the quick review :)

@Wauplin Wauplin merged commit 36d9a3e into main Oct 31, 2023
11 of 16 checks passed
@Wauplin Wauplin deleted the separate-test-and-prod-cache branch October 31, 2023 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants